Vision-Language-Action (VLA) models have advanced autonomous driving, but existing benchmarks still lack scenario diversity, reliable action-level annotation, and evaluation protocols aligned with human preferences. To address these limitations, we introduce DriveAction, the first action-driven benchmark specifically designed for VLA models, comprising 16,185 QA pairs generated from 2,610 driving scenarios. DriveAction leverages real-world driving data proactively collected by drivers of autonomous vehicles to ensure broad and representative scenario coverage, offers high-level discrete action labels collected directly from drivers' actual driving operations, and implements an action-rooted tree-structured evaluation framework that explicitly links vision, language, and action tasks, supporting both comprehensive and task-specific assessment. Our experiments demonstrate that state-of-the-art vision-language models (VLMs) require both vision and language guidance for accurate action prediction: on average, accuracy drops by 3.3% without vision input, by 4.1% without language input, and by 8.0% without either. Our evaluation supports precise identification of model bottlenecks with robust and consistent results, thus providing new insights and a rigorous foundation for advancing human-like decisions in autonomous driving.

Large Language Models (LLMs) have shown significant promise in decision-making tasks when fine-tuned on specific applications, leveraging their inherent common sense and reasoning abilities learned from vast amounts of data. However, these systems are exposed to substantial safety and security risks during the fine-tuning phase. In this work, we propose the first comprehensive framework for Backdoor Attacks against LLM-enabled Decision-making systems (BALD), systematically exploring how such attacks can be introduced during the fine-tuning phase across various channels. Specifically, we propose three attack mechanisms and corresponding backdoor optimization methods to attack different components in the LLM-based decision-making pipeline: word injection, scenario manipulation, and knowledge injection. Word injection embeds trigger words directly into the query prompt. Scenario manipulation occurs in the physical environment, where a high-level backdoor semantic scenario triggers the attack. Knowledge injection conducts backdoor attacks on retrieval augmented generation (RAG)-based LLM systems, strategically injecting word triggers into poisoned knowledge while ensuring the information remains factually accurate for stealthiness. We conduct extensive experiments with three popular LLMs (GPT-3.5, LLaMA2, PaLM2), using two datasets (HighwayEnv, nuScenes), and demonstrate the effectiveness and stealthiness of our backdoor triggers and mechanisms. Finally, we critically assess the strengths and weaknesses of our proposed approaches, highlight the inherent vulnerabilities of LLMs in decision-making tasks, and evaluate potential defenses to safeguard LLM-based decision making systems.

Autonomous Vehicle (AV) technology is advancing rapidly, promising a significant shift in road transportation safety and potentially resolving various complex transportation issues. With the increasing deployment of AVs by various companies, questions emerge about how AVs interact with each other and with human drivers, especially when AVs are prevalent on the roads. Ensuring cooperative interaction between AVs and between AVs and human drivers is critical, though there are concerns about possible negative competitive behaviors. This paper presents a multi-stage approach, starting with the development of a single AV and progressing to connected AVs, incorporating sharing and caring V2V communication strategy to enhance mutual coordination. A survey is conducted to validate the driving performance of the AV and will be utilized for a mixed traffic case study, which focuses on how the human drivers will react to the AV driving alongside them on the same road. Results show that using deep reinforcement learning, the AV acquired driving behavior that reached human driving performance. The adoption of sharing and caring based V2V communication within AV networks enhances their driving behavior, aids in more effective action planning, and promotes collaborative behavior amongst the AVs. The survey shows that safety in mixed traffic cannot be guaranteed, as we cannot control human ego-driven actions if they decide to compete with AV.

Understanding and predicting highway lane-change maneuvers is essential for driving modeling and its automation. The development of data-based lane-changing decision-making algorithms is nowadays in full expansion. We compare empirically in this article different machine and ensemble learning classification techniques to the MOBIL rule-based model using trajectory data of European two-lane highways. The analysis relies on instantaneous measurements of up to twenty-four spatial-temporal variables with the four neighboring vehicles on current and adjacent lanes. Preliminary descriptive investigations by principal component and logistic analyses allow identifying main variables intending a driver to change lanes. We predict two types of discretionary lane-change maneuvers: overtaking (from the slow to the fast lane) and fold-down (from the fast to the slow lane). The prediction accuracy is quantified using total, lane-changing and lane-keeping errors and associated receiver operating characteristic curves. The benchmark analysis includes logistic model, linear discriminant, decision tree, na\"ive Bayes classifier, support vector machine, neural network machine learning algorithms, and up to ten bagging and stacking ensemble learning meta-heuristics. If the rule-based model provides limited predicting accuracy, the data-based algorithms, devoid of modeling bias, allow significant prediction improvements. Cross validations show that selected neural networks and stacking algorithms allow predicting from a single observation both fold-down and overtaking maneuvers up to four seconds in advance with high accuracy.

Game-theoretic motion planners are a potent solution for controlling systems of multiple highly interactive robots. Most existing game-theoretic planners unrealistically assume a priori objective function knowledge is available to all agents. To address this, we propose a fault-tolerant receding horizon game-theoretic motion planner that leverages inter-agent communication with intention hypothesis likelihood. Specifically, robots communicate their objective function incorporating their intentions. A discrete Bayesian filter is designed to infer the objectives in real-time based on the discrepancy between observed trajectories and the ones from communicated intentions. In simulation, we consider three safety-critical autonomous driving scenarios of overtaking, lane-merging and intersection crossing, to demonstrate our planner's ability to capitalize on alternative intention hypotheses to generate safe trajectories in the presence of faulty transmissions in the communication network.

Difficulties in left lane usage are common and exasperating. I'm referring to those darned drivers that sit in the left lane nearly forever, cruising leisurely along without a seeming care in the world, backing up traffic as they do so. You've undoubtedly been stuck behind such a driver. It is exasperating, infuriating, and altogether makes you want to bust a gasket. They get into the left lane and occupy the lane as though it is owned by them. On top of this, they decide to be the unofficial determiner of the allowed speed for the rest of nearby traffic. For example, even though the posted speed limit might be 65 miles per hour, the left lane hog will opt to go at say 55 miles per hour. There are lots of frequently cited reasons or excuses for this type of behavior. One claim is that they are going at the safest appropriate speed. This is based on the logic that the posted speed is the maximum allowed speed, which is not necessarily the safest allowed speed. Indeed, the driver's handbook clearly states that you should never assume that the posted speed is the speed that you are to be driving at.

Its 2030 and a SUV driven by an Autonomous Driving System (ADS) is heading west on a highway. The SUV contains two parents in the front seats and two small children in the back seat. The SUV is going the speed limit of 100 km/hour. The SUV drives through a tight corner and as the SUV makes the final turn a large bull moose weighing over six hundred kilograms shambles onto the road. The autonomous driving system driving the SUV was trained to select the best alternative out of as set of possible outcomes and so the SUV abruptly swerves into the left lane currently occupied by a small sedan going the same speed as the SUV. The SUV ADS had determined that saving the lives of two adults and two children was the greater good even though there was a significant risk that the small sedan would be forced into oncoming traffic travelling East putting the two adult occupants at mortal risk.

Our personalities impact almost everything we do, from the career path we choose to the way we interact with others to how we spend our free time. But what about the way we drive--could personality be used to predict whether a driver will cut someone off, speed, or, say, zoom through a yellow light instead of braking? There must be something to the idea that those of us who are more mild-mannered are likely to drive a little differently than the more assertive among us. At least, that's what a team from MIT's Computer Science and Artificial Intelligence Laboratory (CSAIL) is betting on. "Working with and around humans means figuring out their intentions to better understand their behavior," said graduate student Wilko Schwarting, lead author on the paper published this week in Proceedings of the National Academy of Sciences. "People's tendencies to be collaborative or competitive often spills over into how they behave as drivers.

I love a good road trip. I've spent hundreds of thousands of miles in cars during my life, and the best times were when I knew it would be hours or even days before I reached my destination. Typically a friend (or friends) or family members would accompany me, but on a few occasions, it was just me, my music collection -- and scenery screaming past me at 70 miles per hour. In the past few years, more and more automakers have created semiautonomous systems so that you're no longer "alone" on these drives. One of the more robust (and most famous) is Tesla's Autopilot.

If you've been driving your Tesla in the past week, you're likely enjoying the major upgrade Elon Musk's automaker just issued with a free, over-the-air software update. And if you believe the blog post trumpeting the advance, you've taken a major step towards chillaxing on the highway while the car handles the traffic for you. By accepting this download, owners give their cars the ability to "Navigate on Autopilot," which Tesla says "guides a car from a highway's on-ramp to off-ramp, including suggesting and making lane changes, navigating highway interchanges, and taking exits." It comes with a mass of caveats though, not least that the driver still has to stay in control, and confirm every single move the computer comes up with. No surprise, owners are already debating how much better their cars really are now.